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Abstract 

In this paper we study the asymptotic behaviour of empirical pro- 
cesses when parameters are estimated, assuming that the underlying 
sequence of random variables is long-range dependent. We show com- 
pletely different phenomena compared to i.i.d. situation, as well as 
compared to ordinary empirical processes of long range dependent 
sequences. Applications include Kolmogorov-Smirnov and Cramer- 
Smirnov-von Mises goodness-of-fit statistics. 
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1 Introduction and statement of results 

Let {ei,i > 1} be a centered sequence of i.i.d. random variables. Consider 
the class of stationary linear processes 



oo 

E 

k=0 



We assume that the sequence c^, > 0, is regularly varying with index —(3, 
P G (1/2, 1) (written as Ck € RV-fs). This means that Ck ~ k~^LQ[k) as /c — > 
oo, where Lq is a slowly varying function at infinity. We shah refer to all such 
models as long range dependent (LRD) linear processes. In particular, if the 
variance exists, then the covariances pk '■= EXoX^ decay at the hyperbolic 
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rate, pk = L{k)k-'-^^-^'> =: L{k)k-^, where limk^oo L{k) / Ll{k) = B{2f3 - 
1,1-/3) and B{-, ■) is the beta-function. Consequently, the covariances are 
not summable (cf. [9]). 

Assume that Xi has a continuous distribution function F. Given Xi, . . . , X, 
let Fn{x) = J27=i ^{Xi<x} be the empirical distribution function. 

Assume that Eel < Let r be an integer and define 

n r 
Yn,r = Yl J2 Il'^js'^i-js, ri>l, 

i = l l<jl<---<jr S = l 

SO that Ynfl = n, and y„,i = J2i=i ^i- If P < (2/? - then 

<,:=Var(y„,p)~n2-*'(2/^-i)Lg^(n). (2) 
From [10] we know that for p < (2/3 — 1)""*^, as n — > oo, 

where Zp is a random variable which can be represented by appropriate 
multiple Wiener-Ito integrals. In particular, Zi is standard normal. 

In the present paper we study the asymptotic behaviour of empirical 
processes when unknown parameters of the underlying distribution func- 
tion are estimated. The motivation to study such problems comes from 
Kolmogorov-Smirnov type statistics. From [10] we know that, as n — > oo, 

a-\nsup\Fn{x) - F{x)\ -i |Zi|sup/(x), (4) 

where Zi is a standard normal random variable and / is the density function 
of F. The above result can be used, in principle, to test whether data 
Xi, . . . , Xn are consistent with a given distribution F. If however F belongs 
to a one-parameter family {F{-,9), 6 £ R} say, then in order to use ^ one 
needs to know the value of the parameter 9. A straightforward procedure 
would be to estimate it and use the statistic 

a~\n sup \Fn{x) - F{x; 0n)\, 
xeR 

where F(x;9n) is the distribution function F{x) = F{x;9) in which the 
parameter 9 has been replaced with its estimator However, in the i.i.d. 
case, it is known that such procedure changes a limiting process. To be more 
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specific, assume for a wfiile tfiat Xi, . . . , Xn are i.i.d. random variables and 
consider 

\/n sup \ Fn{x) — F{x)\. 

As it is well-known, the above supremum converges in distribution to the 
supremum of a Brownian bridge on [0, 1]. On the other hand, for a large 
class of estimators, 

V7i\Fnix)-F{x;9n)\, 

converges weakly to a Gaussian process, but no longer to a Brownian bridge. 
The corresponding comments apply to the Cramer-Smirnov-von Mises statis- 
tic 

I {Fn{x)-F{x)fdF{x) 
JR 

and its 'estimated' version 

[ {Fn{x)-F{x-9n)fdF{x-X)- 

We refer to [5], [8], [11] and [1] for more details. 

Coming back to LRD sequences, we will focus on a location-scale family 
of distributions. We shall assume that Yi = aXi+fj,, where Xi is given by ([T|) 
and cr 7^ 0. Clearly, if F is the distribution of Xi and H is the distribution 
of Yi, theni?(x) =f(^). Moreover, the empirical processes 

(3n{x) = a-}^n{Fn{x) - F{x)), x e M 

and 

7„(x) = a~^^n{Hn{x) - H{x)), x G M 
associated with Xi and Yi, respectively, are related by 

7n(x) = Pn (^) . (5) 

From [To], (3n{x)^f{x)Zi, so that 7„(x)^/(^-^)Zi. Here and in the sequel, 
^ denotes weak convergence in !)((— cxd, oo)). On the contrary, if On is an 
appropriate sequence of estimators of the mean ^u, we will show that, as 
n — > oo, 

7„(x) = (J^jn(i?„(x) - H{x] 9n)), a: G R 

converges in probability to 0. Choosing a different scaling one can obtain 
weak convergence, however the limiting process depends on the choice of 
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the estimator. In particular, using On = Yn (the sample mean of Yi, . . . , 1^) 
or On = Mn (M-estimator), we can obtain different limits, depending on 
the so-called second-order M-rank of the estimator M„ introduced in |12j . 
Also, the scaling and the limiting process depend on whether /3 > 3/4 or 
/3 < 3/4. In particular, if /3 > 3/4, then we obtain y^-consistency of a 
modified Kolmogorov-Smirnov type statistics. The appropriate results are 
stated in Theorems 11.21 and 11.41 

The proofs of our results will be based on a reduction principle for long- 
range dependent empirical processes (see Theorem 11.11 below), combined 
with approximation method as in [Ij. The fact, that we were able to use the 
latter, Hungarian-like approach, shows its extreme power. The Hungarian 
construction approach was for example employed to obtain the Komlos- 
Major-Tusnady (KMT) strong approximation of empirical processes. Then, 
this approach was followed to establish a number of optimal or almost opti- 
mal results for functionals of empirical and quantile processes, including the 
one in [Ij for empirical processes with parameters estimated (we refer to [2]). 
The KMT construction is tailored for the i.i.d. situation. However, a lot 
of further developments based on this kind of approach, can be applied to 
long-range dependent sequences. Very recent examples of such an approach 
include [3], [1], [H]. 

The reduction principle was obtained first in [6j in case of subordinated 
Gaussian processes. In more generality, it was obtained in the landmark 
paper [10]; see also [13] for related studies. The best available result along 
these lines is due to Wu [15]. To state a particular version of his result, we 
shall introduce the following assumptions, which will be valid throughout 
the paper. Let be the distribution function of the centered i.i.d. sequence 
> 1} • Assume that for a given integer p, the derivatives F^^^ , • • • , F^'^'^^^ 
of -Ff are bounded and integrable. Note that these properties are inherited 
by the distribution F as well (cf. [10] or [15j). 

Theorem 1.1 Let p be a positive integer. Then, as n ^ oo, 

p 



E sup 

where 



J2ihx.<.} - F{x)) + 5:(-i)-^FW(x)y„ 

1=1 r=l 



2 

= 0(H„ + n(logn)2), 



^ _ / 0{n), (p + l)(2/3-l) > 1 

\ 0(n2-{P+i){2/3-i)Lf +^)(n)), (p + l)(2/3-l)<l ■ 

We will a require second-order expansion, thus in the above theorem, p = 2. 
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Let V' be a real- valued function of bounded variation such that Exp{Yi — /x) 
0. M-estimators are defined as 

M = Mn = arg min 

For A: = 1,2, let 

Jn 

Let k* = k*{P) = [1/(2/3 — 1)], where [•] denotes the integer part. The 
second-order rank rM(2) of the M-estimator is: rAf(2) = 2 if /c* = 1 (so 
that P > 3/4); rM(2) = 2 if /c* > 1 and A2 / 0; rAf (2) > 2 if fc* > 1 and 
A2 = 0. We refer to [12] for more details. 

Let 

Now, we are ready to state our results. We start with the case /3 < 3/4. 

Theorem 1.2 Assume that 9q = ^ and (3 < 3/4. Then, under the condi- 
tions of Theorem ll.il as n —> 00, we have 

• If 9n = Yn or On = M„, then 

sup |7„(a;)| = op(l). (6) 

• If 9n = Yn, then 

a-^n{x) = <>(i/„(x) - H{x; ^"n))^/^^) V, (7) 

where V is a linear combination of Z2 and ^Zf. 

• If9n = Mn, Ee^^^'"'*^^^ < 00 and rM{2) > 2, then (HD holds. 

• If9n = Mn, Ee^^^''*^^^ < 00 and rM{2) = 2 

a-'%{x) = a-ln{Hn{x)-H{x;9n)Hf^'^ 

\ a / zAi a \ a 

(8) 

where V is as in ([7|) and Vi is a linear combination of Zf and Z2 ■ 



n 






,x£ ml 
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Example 1.3 Assume that /i = 0, / is symmetric and ip is skew-symmetric. 
For (3 < 3/4, rM(2) > 3 (cf. [12]) and the hmiting behaviour is described 
by ©• If> however, / is not symmetric, then A2 7^ and dH) holds. 

As for the case (3 > 3/4 we have the following theorem. 

Theorem 1.4 Assume that 9q = fi and (3 > 3/4. Then, under the condi- 
tions of Theorem 11.11, as n ^ 00, we have 

• IfOn = Yn or On = M„, then 

sup \%{x)\ = Op(l). 

• If 6n = Yn, then 

^<yn,in-^n{x) = V^{Hn{x) - H{x; en))^W (^^) , (9) 
where W{-) is a Gaussian process. 

• //4 = Mn, Eet^^''*^^^ < oo, then 

2 

m 

(T^ is given by the formula (1-18) in |12j . 

An immediate corollary to Theorem ll.2l is the following Cramer-Smirnov-von 
Mises test. An appropriate version can also be stated in terms of Theorem 
Ol 

Corollary 1.5 Let Oq = fi and On = Yn- Under the conditions 0/ Theorem 

Mi 

<yn\n l^{Hn{x)-H{x;On)fdH{x;0n) ^ 1^1^ (^^^y / (^^) dx. 

The above result should be compared with a regular situation of non-estimated 
Cramer-Smirnov-von Mises statistics in [7]. The limiting distribution for the 
model ([I]) in case of Gaussian errors e^, is a random variable Zf multiplied 
by a deterministic function. 

In what follows C will denote a generic constant which may be differ- 
ent at each of its appearance. Also, for any sequences a„ and 6„, we write 
On ~ bn if liuin^ooO^n/bn = 1- Moreover, /^^^ denotes the A;th order deriva- 
tive of /. 
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2 Proofs 



Let p be a positive integer. Recall that 
and let 

dr, 



n-(i-^)Lo^(n)(logn)5/2(iQgiQg^)3/4^ (p+l)(2/3_i) > i 
n-P('3-|)^P(j^)(logn)i/2(iogiog^)3/4^ (p+ l)(2/3- 1) < 1 



Note that dn,2 = o(fln) provided /? < |, 



Put 



n 



P 
r=l 



Y.{l{X,<a,}-F{x))+Vn,pi^ 



i=l 



Using Theorem ll.il we obtain 

a~]p sup |S'n,p(2;)| 



Oa...(^"^^"^^^"^^^ VN(logri)^/2(iogiog^)3/4)^ (p+ l)(2/3 - 1) > 1 
Oa.s.(»^~^^"^^^oH(logn)i/2(ioglogn)3/4), (p+l)(2/?-l) < 1 



Since (see ([2])) 



we obtain 



(11) 



sup |/?n(2;) + <T„ iK,p(a:) 



(12) 



n,p 



sup 



i=l 



Oa.s. {dn,p) • 



For a function (7(x; 0) denote by Vgg{x] Oq) its rth order derivative with 
respect to 6, evaluated at 6* = Oq. In particular, V = V^. 
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2.1 Proof of Theorem D 

Recall (0). For an arbitrary unknown parameter Oq and its estimator On we 
have by 

%{x) = 7„(x) + a~\n{H{x; 0o) - H{x] 9n)) 



X — fl 



a 



+ a-\n{H{x;eo)-Hix;9n)) 



Op{d 



X — fj, 



+ <]n(0o - en)VeH{x; 



Op{dn,2) - (^n!lf ^ ^ 



a 



i=l 



X — /i 

a 



n,2 



+lcj-^in{eo-9nfvlH{x;9:), 

with some 0* such that |0* — 6'„| < \9o — 9'^\. 
If ^0 = A*! then 

Also, if 9n = Yn, then 

9n — 9o = aXn 
Hence, using uniform boundness of /^^\ 

7n(x) = - a-}/ f ^) X: + ^,1/^'^ (^) Yn,2 + 



(13) 

(14) 
(15) 



(T 



) E + ^-.-,1-/^^^ ( + Op (a->X3) . 



Since /3 < 3/4, note that o"„^iy„^2 = Op(l) (cf- dSl)), cj„|nX^ = op(l) and 
a~\nX^ = op(l). Thus, we conclude that sup^ |7ra(2;)| ^ for 9^ = Yn- 



Further, 
-1 



an sup 



X — 



a 



Op((in,2an^) + Op(a„ V„inX;^) = Op(l) + Op{an^an]xnn ^a\^) 

Op(l). 
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Thus, dZD follows. 

If On = Mn then, as in ([13]) and ([HD, 

n 



ln{x) = Op{dn,2) - CfJJ ^ ^ 



a 



1=1 



X — fi 
a 



1 _x ^ o \ f ^ ~ 



a 



a 



Yn,2 + 
X — fi 



a ' V c 

-i„ 



+ 



ifx - MnY + Op{a-\n{ii - M, 



X — fj. 



a 



Yn,2 - -Cy7A<yn - Mn)f 



a 



X — fi 



a 



Mnr + Opia-\nifi-Mnr) 



From [12], 



a^^in{Mn - fi) = fT„ in(y„ - fi) + op(l) ci^Zi 



(16) 



and (T„ luiYn — Mn) = op{l). Thus, sup^, |7n(2;)| for 0„ = 

If rM(2) > 2, then from [H] Theorem 1.1], 

a-V-ln(y„ - M„) = op(l), 

thus in this case 



a„ ^ sup 



X — fi 



1 



Op{dn,2a-^) + op(l) + Op{a-^(j-\n{fi - Mnf) = op(l). 



Therefore, in view of (116]) . ([7]) follows. 

If (2) = 2, then a~-'^cj~5^n is the proper scaling for {Yn — M„) and thus 



On ^ sup 



7n(x) - /(I) 



X — fi 



a 



<^n\Yn,2 + 



X — fi 

a 



{Yn - Mn) 



= Op{dn,2an^) + Op(a„ - Mnf) = Op(l), 

and hence ([8]) follows using ([T6]) and Corollary 1.1 in fT2]. 
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2.2 Proof of Corollary [TSl 

Write 

J %{xfdH{x;0n) = j %{xfh{x;eo)dx 
+ J %{xf{h{x;0n - h{x;6o))dx. 
As for the second term, we have 

J %{xfVeKx; eo){9n) - eo)dx + 

where Rn = Op{{9n — ^o)^) = op{9n — 6q). Thus, the second term is of a 
smaller rate than the first one and the limiting behaviour of J 7„(x)^d//(x; 0„) 
is the same as that of / ^n{x)'^h{x;6Q)dx. Thus, Corollarv 11.51 follows from 
Theorem 11.21 





2.3 Proof of Theorem [13] 

Recall that /3 > 3/4. Then 



a 



x — II 



a 



F 



X — fl 



f 



+ Vni^F 
X — ^ 



X — 



a 



X — fJ. ^ 



Wr.. 



a 

X — fl 
a 



V^ieo - On)f 



i=l 
X — fl 



n a \ (J 

n 



I 



X — fl'" 



n a 



a 



If ^0 = I-'' and On = Yn, then via (fT5 



sup 



nan,in 7„(x) - Wn 



X — fl 



a 



Op{V^{fl-enf)=Op{l). 



Thus, using [151 Theorem 3], we obtain ([9]) 
If 6q = fl and 0„ = M„, then 



sup 



V^(yn,in-^ln{x) - Wn{x) + -f (- ] ^/^(M„ - Yn 

a \ a 



Op(l). 
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If /3 > 3/4, then from [HI Theorem 1.1], ^/n{Mn - F^) ^ ^(0,cJ^). Thus, 
([TU]) follows. 

© 
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